Repository navigation
fix(compose): bound the bundled MySQL binary log - #1196
Open
linus-liu-web wants to merge 1 commit into
Open
linus-liu-web wants to merge 1 commit into
linus-liu-web wants to merge 1 commit into
Conversation
The default Compose stack starts MySQL with binary logging on and no retention, so every checkpoint, telemetry and log row is also written to the binlog without an upper bound. A write storm then fills loopforge_mysql-data with binary logs while the database itself stays small: Tencent#1138 reported 298 GB of binlogs in four days while the largest table was 0.16 GB, and a Compose deployment here accumulated 1.18 GB of binlogs in about a day. Bound the log instead of leaving it unbounded: - --binlog-expire-logs-seconds=86400 keeps one day of binary logs; - --max-binlog-size=67108864 rotates every 64 MiB. Deployments that need point-in-time recovery or replication can raise or drop the two flags; single-node evaluation stacks can pass --skip-log-bin. Document both, plus the PURGE BINARY LOGS command for installations that already accumulated logs.
2 tasks done
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What changed
The default Compose stack starts
mysql:8.4with binary logging on and no retention. Every checkpoint, telemetry and log row the platform writes is also written to the binary log with no upper bound, so a write storm fillsloopforge_mysql-datawith binlogs while the database itself stays small. #1138 reports 298 GB of binlogs in four days with the largest table at 0.16 GB; a Compose deployment here accumulated 1.18 GB in about a day.Bound the log instead of leaving it unbounded:
--binlog-expire-logs-seconds=86400keeps one day of binary logs;--max-binlog-size=67108864rotates every 64 MiB.Deployments that need point-in-time recovery or replication can raise those values or remove the flags; single-node evaluation installations that need no recovery can pass
--skip-log-bin. The deployment guide documents both choices and thePURGE BINARY LOGS/SET PERSIST binlog_expire_logs_secondscommands for installations that already accumulated logs.Related: #1138 (binary log growth from the workspace checkpoint loop; the engine-side retry/lease fix is #1148 and explicitly leaves this MySQL/Compose configuration aside). This PR only bounds the default; it does not claim to fix the loop itself.
Scope
Compatibility
No application behavior change: the database schema, DSN and data are untouched. Existing volumes keep working; MySQL applies the new retention/size limits on restart. Operators who rely on binary logs for point-in-time recovery or replication must raise the values or remove the flags, as documented. MySQL 8.4 has binary logging enabled by default, so the flags only bound it.
Validation
docker compose --env-file <existing .env> -f compose.yaml --profile platform configrenders the new MySQL command unchanged.mysql:8.4with the exact new flags and confirmedSHOW VARIABLES:binlog_expire_logs_seconds=86400,max_binlog_size=67108864,log_bin=ON. Container removed afterwards.mysql-datavolume (the flags are read at startup and do not require reinitialization).Safety